Image sensing with a display

ABSTRACT

An electronic device includes a cover glass having a display surface, a pixelated photoemitting element array, and a pixelated photodetecting element array. The pixelated photoemitting element array emits a light signal through the cover glass to the display surface. The pixelated photodetecting element array is positioned relative to the pixelated photoemitting element array and the cover glass to receive a reflected light signal. The reflected light signal includes a portion of the emitted light signal reflected by total internal reflection from a refractive boundary at the display surface. Operation of each pixel is switched between the one or more photodetecting elements and the one or more photoemitting elements by the pixel selector signal component received from the pixel selector signal bus. A sensing trigger is configured to trigger the imaging scan by the pixelated photoemitting element array and the pixelated photodetecting element array, responsive to detection of an initiating action.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims benefit of priority to U.S. ProvisionalPatent Application No. 62/304,853, entitled “High-Resolution Imaging andSensing with Display” and filed on Mar. 7, 2016, which is specificallyincorporated by reference for all that it discloses and teaches.

The present application is also related to U.S. patent application Ser.No. ______ [Docket No. 360505.01], entitled “Pixel having a Photoemitterand a Photodetector Triggered by a Pixel Selector Signal Bus,” and U.S.patent application Ser. No. ______ [Docket No. 360508.01], entitled“Triggered Image Sensing with a Display,” both of which are filedconcurrently herewith and are specifically incorporated by reference forall that they disclose and teach.

BACKGROUND

Fingerprint detection systems for use with computing devices may employa variety of technologies, including capacitive sensing, ultrasoundsensing, lensed digital cameras, etc. However, such solutions come withsignificant limitations. For example, bezel-less or very small bezeldevices do not leave sufficient area for fingerprint detectioncomponents outside of the display area. Furthermore, capacitive sensingis very sensitive to the distance between the finger and the sensor,such that the cover glass of a display of a computing device maydramatically reduce the effectiveness of the capacitive sensingresolution if the capacitive sensing components are positioned beneaththe display. Ultrasonic sensing is accompanied by noise issues andmanufacturing issues (including detrimental mechanical impedance betweenthe sensor and the display surface). Lensed digital cameras tend to bebulky and expensive. Many such solutions also tend to be difficult toscale in area across the area of the computing device front face ordisplay.

SUMMARY

The described technology provides an electronic device including a coverglass having a display surface, a pixelated photoemitting element array,and a pixelated photodetecting element array. The pixelatedphotoemitting element array has one or more selected photoemittingelements of the pixelated photoemitting element array configured to emita light signal through the cover glass to the display surface. Thepixelated photodetecting element array is positioned relative to thepixelated photoemitting element array and the cover glass to receive areflected light signal at individual photodetecting elements of thepixelated photodetecting element array. The reflected light signalincludes a portion of the emitted light signal reflected by totalinternal reflection from a refractive boundary at the display surface ofthe cover glass. Image processing circuitry is electrically coupled tothe pixelated photoemitting element array and the pixelatedphotoemitting element array and is configured to stitch the reflectedlight signal received by each photodetecting element of the pixelatedphotodetecting element array into a composite image of an object incontact with the display surface of the display.

In another implementation, an electronic device includes a cover glassof a display having a display surface, a pixel selector signal busconfigured to communicate a pixel selection signal component; and apixel array of the display. The pixel array is electrically connected tothe pixel selector signal bus and includes multiple pixels configured tosense an image of an object in contact with a surface of the display.Each pixel in the pixel array includes one or more photodetectingelements and one or more photoemitting elements. Operation of each pixelis switched between the one or more photodetecting elements and the oneor more photoemitting elements by sensor control instructions in thepixel selector signal component received from the pixel selector signalbus. Image processing circuitry is electrically coupled to the pixelarray and is configured to scan light from multiple pixels of the pixelarray. The image processing circuitry stitches a light signal reflectedfrom a refractive boundary at the display surface and received byphotodetecting elements of the pixelated photodetecting element arrayinto a composite image of the object.

In another implementation, an electronic device includes a cover glasshaving a display surface, a pixelated photoemitting element array, and apixelated photodetecting element array. One or more selectedphotoemitting elements of the pixelated photoemitting element array isconfigured to emit a light signal through the cover glass to the displaysurface as part of an imaging scan. The pixelated photodetecting elementarray is positioned relative to the pixelated photoemitting elementarray and the cover glass to receive a reflected light signal atindividual photodetecting elements of the pixelated photodetectingelement array as part of the imaging scan. A sensing trigger isconfigured to trigger the imaging scan by the pixelated photoemittingelement array and the pixelated photodetecting element array, responsiveto detection of an initiating action. Image processing circuitry iscoupled to the sensing trigger, the pixelated photoemitting elementarray and the pixelated photodetecting element array. The sensingtrigger transmits trigger data to image processing circuitry to initiatethe imaging scan through the cover glass.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

Other implementations are also described and recited herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example image sensing display of an electronicdevice.

FIG. 2 illustrates another example image sensing display of anelectronic device.

FIG. 3 illustrates example photoemitter and photodetector pixelconfigurations for an image sensing display of an electronic device.

FIG. 4 illustrates an example fingerprint scanning process and anexample resulting image.

FIG. 5 illustrates an example schematic of an image sensing system for adisplay of an electronic device.

FIG. 6 illustrates an example use of total internal reflection (TIR) forimage sensing with a display of an electronic device.

FIG. 7 illustrates an example pixel array, wherein each pixel includes aphotoemitting element and a photodetecting element occupying distinctareas of the pixel.

FIG. 8 illustrates an example pixel array, wherein each pixel includes aphotoemitting element and a photodetecting element occupying overlappingareas of the pixel.

FIG. 9 illustrates a schematic of an example pixel drive electroniccircuit for a pixel having one or more photoemitting elements in aphotoemitter and one or more photodetecting elements in a photodetector.

FIG. 10 illustrates a schematic of an example triggered image sensingsystem for a display of an electronic device.

FIG. 11 illustrates example operations for image sensing with a display.

FIG. 12 illustrates example operations for switching a pixel betweenphotoemitting mode and photodetecting mode.

FIG. 13 illustrates example operations for triggered image sensing witha display.

FIG. 14 illustrates an example processing system for use in imagesensing and/or triggered image sensing on a display.

DETAILED DESCRIPTIONS

FIG. 1 illustrates an example image sensing display 100 of an electronicdevice 102. In FIG. 1, the example electronic device 102 is shown as amobile phone, although other electronic devices may include withoutlimitation tablet computers, electronic displays, laptop computers,all-in-one computers, electronic accessories, building security devices,automated teller machines, etc. A user's thumb 104 is shown pressedagainst a display surface 110 of the image sensing display 100.Fingerprint sensing in these and other environments is an example ofimaging that can be provided by the described technology, although othertypes of imaging may be employed.

As shown in the blown-up drawing in circle 106, ridges 108 on the user'sthumb 104 contact the display surface 110 of the image sensing display100, which in at least one implementation includes a cover glass 112, apolarizing layer 114, one or more quarter wavelength plate(s) 116, andan photodetecting/emitting layer 118. In one implementation, thephotodetecting/emitting layer 118 may include multiple sublayers andorganic light-emitting diodes (OLEDs) positioned on a substrate 119. AOLED sublayer may include transparent or translucent regions, which canallow light to pass through sublayers of the photodetecting/emittinglayer 118. In one implementation, as shown in FIG. 1, at least onephotodetecting element and one or more separate photoemitting elementsare included in a single pixel. In an alternative implementation, anindividual OLED element may be selectively biased to operate as either aphotodetecting element or a photoemitting element. Other electronicdevice components 117 are shown immediately below the image sensingdisplay 100, although intervening layers may exist.

In one implementation, the photodetecting/emitting layer 118 includes anarray of pixels, such as pixels 122, wherein each pixel includes atleast one photoemitting element, such as photoemitting elements 126, andat least one photodetecting element, such as photodetecting elements124. In the photodetecting/emitting layer 118, each pixel is shown asincluding three photoemitting elements (e.g., red, green, and blueelements) and a photodetecting element, although other configurationsare contemplated. As shown, the photodetecting elements andphotoemitting elements of each pixel are configured in substantially thesame sublayer or plane. By configuring a single pixel to includephotodetecting elements and photoemitting elements in the same sublayer,the image sensing display 100 can provide both photodetecting andphotoemitting functionality without substantially increasing thethickness of the image sensing display 100.

In the example of FIG. 1, light emitted by one of the photoemittingelements 126 is transmitted to a refractive boundary at the displaysurface 110 of the image sensing display 100. Light having an angle ofincidence greater than a critical angle is reflected by total internalreflection through the cover glass of the image sensing display 100.Where a feature of an object (e.g., a ridge of a finger print) isoptically-coupled to the display surface 110, the reflected lightscatters toward the photodetecting/emitting layer 118. Where no featureof the object is optically-coupled to the display surface 110, thereflected light reflects in a non-scattered manner toward thephotodetecting/emitting layer 118. The non-scattered reflected light andthe ridge-scattered reflected light are captured by a photodetectingelement 128. As described in more detail below, the photoemittingelements 126 of array of pixels 122 can scan a region of the displaysurface 110 of the image sensing display 100 to illuminate an object onthe display surface 110 and can capture light reflecting off the displaysurface 110 using the photodetecting elements 124 of the array of pixels122. The captured light can then be stitched together to yield acomposite image of the object.

FIG. 2 illustrates another example image sensing display of anelectronic device. In FIG. 2, the example electronic device 202 is shownas a mobile phone, although other electronic devices may include withoutlimitation tablet computers, electronic displays, laptop computers,all-in-one computers, electronic accessories, building security devices,automated teller machines, etc. A user's thumb 204 is shown pressedagainst a display surface 210 of the image sensing display 200.Fingerprint sensing in these and other environments is an example ofimaging that can be provided by the described technology, although othertypes of imaging may be employed.

As shown in the blown-up drawing in circle 206, ridges 208 on the user'sthumb 204 contact the surface 210 of the image sensing display 200,which in at least one implementation includes a cover glass 212, apolarizing layer 214, one or more quarter wavelength plate(s) 216, andan photodetecting/emitting layer 218. In one implementation, thephotodetecting/emitting layer 218 may include multiple sublayers andorganic light-emitting diodes (OLEDs) positioned on a substrate 219. AnOLED sublayer may include transparent or translucent regions, which canallow light to pass through sublayers of the photodetecting/emittinglayer 218. An individual OLED element may be biased to operate as eithera photodetector element or a photoemitting element. In oneimplementation, as shown in FIG. 2, at least one photodetecting elementand one or more separate photoemitting elements are included in a singlepixel. In an alternative implementation, an individual OLED element maybe selectively biased to operate as either a photodetecting element or aphotoemitting element. Other electronic device components 217 are shownimmediately below the image sensing display 200, although interveninglayers may exist.

In one implementation, the photodetecting/emitting layer 218 includes anarray of pixels, such as pixels 222, wherein each pixel includes atleast one photoemitting element, such as photoemitting elements 226, andat least one photodetecting element, such as photodetecting elements224. In the photodetecting/emitting layer 218, each pixel is shown asincluding three photoemitting elements (e.g., red, green, and blue) anda photodetecting element, although other configurations arecontemplated. As shown, the photodetecting elements and photoemittingelements of each pixel are configured in different sublayers or planes,wherein light reflected by total internal reflection from a refractoryboundary at the display surface 210 can pass through the transparent ortranslucent regions of the photodetecting/emitting layer 218 to aphotodetecting element 224. By configuring a single pixel to includephotodetecting elements and photoemitting elements in differentsublayers, the image sensing display 200 can provide overlappingphotodetecting and photoemitting functionality without substantiallyincreasing the thickness of the display (although typically, theimplementation of FIG. 2 will be thicker than that of FIG. 1) orsubstantially decreasing the resolution and/or illumination of the imagesensing display 200.

In the example of FIG. 2, light emitted by one of the photoemittingelements 226 is transmitted to a refractive boundary at the displaysurface 210 of the image sensing display 200. Light having an angle ofincidence greater than a critical angle is reflected by total internalreflection through the cover glass of the image sensing display 200.Where a feature of an object (e.g., a ridge of a finger print) isoptically-coupled to the display surface 210, the reflected lightscatters toward the photodetecting/emitting layer 218. Where no featureof the object is optically-coupled to the display surface 210, thereflected light reflects in a non-scattered manner toward thephotodetecting/emitting layer 218. The non-scattered reflected light andthe ridge-scattered reflected light are captured by a photodetectingelement 228. As described in more detail below, the photoemittingelements 226 of array of pixels 222 can scan a region of the displaysurface 210 of the image sensing display 200 to illuminate an object onthe display surface 210 and can capture light reflecting off the displaysurface 210 using the photodetecting elements 224 of the array of pixels222. The captured light can then be stitched together to yield acomposite image of the object.

FIG. 3 illustrates example photoemitter and photodetector pixelconfigurations 300, 302, 304, and 306 for an image sensing display of anelectronic device. Each pixel configuration 300, 302, 304, and 306illustrates a single pixel that can be part of a display array of pixelsin an image sensing display. The different pixel configurations 300,302, 304, and 306 provide tradeoffs among display resolution,illumination intensity, image detection resolution, manufacturingdifficulty, etc. Other configurations are also contemplated.

In the pixel configuration 300, a display layer 308 includes a coverglass and other optical sublayers. Various display sublayer combinationsare contemplated. Within the pixel configuration 300, threephotoemitting elements 310, 312, and 314 and a photodetecting element316 are embedded within the display layer 308. Some portion of the lightemitted from the photoemitting elements 310, 312, and 314 passes throughthe cover glass to the display surface 301, is reflected by totalinternal reflection off the refractive boundary at the display surfaceof the cover glass (e.g., scattering off points of optical coupling withfeatures of an object, such as one or more ridges of a finger) back intothe display layer 308 to be captured by one or moredisplay-layer-embedded photodetecting elements (such as element 316) ofthe pixels in the display array. Responsive to a scan of light acrossthe one or more objects, the reflected light captured by thephotodetecting elements is then combined or stitched together to yield acomposite image of the one or more objects.

In the pixel configuration 302, a display layer 318 includes a coverglass and other optical sublayers. Various display sublayer combinationsare contemplated. Within the pixel configuration 302, threephotoemitting elements 320, 322, and 324 are embedded within the displaylayer 318 and a photodetecting element 326 is bonded to the surface 311of the display layer 318 that is opposite to the display surface 303 ofthe display layer 318. Some portion of the light emitted from thephotoemitting elements 320, 322, and 324 passes through the cover glassto the display surface 303, is reflected by total internal reflectionoff the refractive boundary at the display surface 303 of the coverglass (e.g., scattering off points of optical coupling with features ofan object, such as one or more ridges of a finger) back into the displaylayer 318 to be captured by one or more display-layer-bondedphotodetecting elements (such as element 326) of the pixels in thedisplay array. Responsive to a scan of light across the one or moreobjects, the reflected light captured by the photodetecting elements isthen combined or stitched together to yield a composite image of the oneor more objects.

In the pixel configuration 304, a display layer 328 includes a coverglass and other optical sublayers. Various display sublayer combinationsare contemplated. Within the pixel configuration 304, threephotoemitting elements 330, 332, and 334 are bonded to the surface 313of the display layer 328 that is opposite to the display surface 305 ofthe display layer 328 and a photodetecting element 336 is embeddedwithin the display layer 328. Some portion of the light emitted from thephotoemitting elements 330, 332, and 334 passes through the cover glassto the display surface 305, is reflected by total internal reflectionoff the refractive boundary at the display surface of the cover glass(e.g., scattering off points of optical coupling with features of anobject, such as one or more ridges of a finger) back into the displaylayer 328 to be captured by one or more display-layer-embeddedphotodetecting elements (such as element 336) of the pixels in thedisplay array. Responsive to a scan of light across the one or moreobjects, the reflected light captured by the photodetecting elements isthen combined or stitched together to yield a composite image of the oneor more objects.

In the pixel configuration 306, a display layer 338 includes a coverglass and other optical sublayers. Various display sublayer combinationsare contemplated. Within the pixel configuration 306, threephotoemitting elements 340, 342, and 344 and a photodetecting element346 are bonded to the surface 315 of the display layer 338. Some portionof the light emitted from the photoemitting elements 340, 342, and 344passes through the cover glass to the display surface 307, is reflectedby total internal reflection off the refractive boundary at the displaysurface of the cover glass (e.g., scattering off points of opticalcoupling with features of an object, such as one or more ridges of afinger) back into the display layer 338 to be captured by one or moredisplay-layer-bonded photodetecting elements (such as element 346) ofthe pixels in the display array. Responsive to a scan of light acrossthe one or more objects, the reflected light captured by thephotodetecting elements is then combined or stitched together to yield acomposite image of the one or more objects.

Other configurations may also be employed including differentphotodetecting/photoemitting element ratios (the ratio illustrated inFIG. 3 shows three photoemitting elements for each photodetectingelement), different combinations of display sublayers, etc. In oneimplementation, photodetecting elements and photoemitting elements of asingle pixel are controlled by a shared bus through control circuitry.The control circuitry selectively turns on and off individualphotodetecting/photoemitting elements of one or more pixels based on thecontrol signals received on the shared bus.

FIG. 4 illustrates an example fingerprint scanning process 400 and anexample resulting image 402. In a first stage 404, individual rows (orcombinations of rows) of photoemitting elements are turned on in ascanning mode (as represented by the arrows), such that emitted light405 scans across an image detection region 406 of the display surface.Some portion of the scanning light passes through the display surface,where it selectively reflects from the refractive boundary at thedisplay surface (scattering off points of optical coupling with featuresof an object, such as one or more ridges 407 of a finger, on the displaysurface) back through the display's cover glass for capture byphotodetectors embedded in the display or bonded to a surface of thedisplay.

The first stage 404 shows a progression of illuminating light 405emitted from photoemitting elements in display pixels scanning acrossthe image detection area 406. Some of the illuminated light isselectively reflected from the refractive boundary at the displaysurface (scattering off points of optical coupling with features of anobject, such as one or more ridges 407 of a finger, on the displaysurface) back through the display's cover glass for detection byphotodetectors. In one implementation, captured light is captured infront of the scanning row as it scans across the image detection region406. In a second stage 408, the captured light signals are recorded inmemory, high pass filtered (e.g., using a Fast Fourier Transform (FFT)filter to remove noise from the image), and stitched together to form acomposite image 402 of the object features. In one implementation,portions or blocks of the captured reflected light are selected based onthe signal-to-noise ratio for use in the stitching process.

Stitching involves combining multiple captured light signals havingoverlapping fields of view to generate a high-level image having an arealarger than any single field of view of the photodetecting elements. Forexample, the reflected light from the image detection region 406 iscaptured by multiple photodetectors having a small field of view ascompared to the overall area of the image detection region 406, and thecaptured light is combined into a resulting image 402 that representsmost of the image detection region 406. The resulting image 402substantially reproduces the image features that contact the displaysurface (in the example, reproducing the fingerprint ridges 407).

It should be understood that scanning, whether for illumination orphotodetection, shapes need not be limited to lines, columns or rows. Inone implementation, scanning is performed, for either or both ofillumination and photodetection, in blocks, rings, etc. Furthermore, inone implementation, partial images that are roughly in a ring shape aredetected by the photodetectors and stitched together to form theresulting composite image. Example rings may be a millimeter or two inouter diameter, centered on a single photoemitting element, althoughother implementation may be employed.

FIG. 5 illustrates an example schematic of an image sensing system 500for a display of an electronic device. The display includes a coverglass 502 for which at least a portion of the display area is occupiedby a photoemitter/detector array 504 (e.g., any remaining display maymerely be occupied by a photoemitter array, without any photodetectors).Responsive to control signals 516 provided by image processing circuitryand software 508, photoemitting elements in the photoemitter/detectorarray 504 emit scanning light through the cover glass 502. Some portionof the scanning light is reflected from a refractive boundary at thesurface 510, and some portion of the reflected light is scattered offobject features 514 optically-coupled at the surface 510 of the coverglass 502. The reflected light, including the feature-scattered light,is captured by photodetecting elements in the photoemitter/detectorarray 504 and transmitted to the image processing circuitry and software508.

In one implementation, the photoemitter/detector array 504 includes a300 ppi sensor array having a sensing area of 1.0″×0.8″. At 8bits/pixel, the photoemitter/detector array 504 can be calculated tocapture 576,000 bits/frame (i.e., (300×1.0)×(300×0.8)×8), although otherphotoemitter/detector array 504 configurations may be employed. Assumingthe above-described 576,000 bits/frame metric and a 60 frame/sec scanrate, the photoemitter/detector array 504 provides a data rate of34,560,000 bits/sec of scanned image data 506 being sent to the imageprocessing circuitry and software 508, which stitches the scans ofcaptured light together into a composite image representing the scannedobject (e.g., a fingerprint).

FIG. 6 illustrates an example use 600 of total internal reflection (TIR)for image sensing with a display 602 of an electronic device. Thedisplay 602 includes a photodetecting/emitting layer 604 having apixelated photodetecting element array (such as an array includingphotodetecting elements 624) and a pixelated photoemitting element array(such as an array including a photoemitting element 625). Pixelatedrefers to dividing an array into individual pixels for display and/orphotodetection in a digital format. In some cases, a pixel in apixelated display may be referenced based on a column and/or rowselection, although other referencing frameworks may be employed.

The example use 600 depicts light emitted from a photoemitting element606 of the pixelated photoemitting element array toward a displaysurface 608 of a cover glass of the display 602. The light strikes thedisplay surface 608, internal to the cover glass, wherein the displaysurface 608 operates as a medium boundary (or refractive boundary)between the cover glass and the atmosphere outside the electronicdevice.

When a propagating light wave strikes a refractive boundary, the wave'sinteraction with that boundary can vary depending on the relativerefractive indices of the materials on each side of the refractiveboundary and on the wave's angle of incidence (i.e., the angle at whichthe light wave strikes the refractive boundary with respect to thenormal to that boundary—see θ and the normal 610). In the case of thedisplay 602, the refractive index of the display's cover glass isgreater than the refractive index of the atmosphere outside the display.Accordingly, if the light wave's angle of incidence is less than thecritical angle θ_(C) of the refractive boundary, some of the light wavewill pass through the refractive boundary and some of the light wavewill be reflected back into the cover glass. (The critical angle θ_(C)is dependent upon the relative refractive indices of the materials oneach side of the refractive boundary, according to Snell's Law.) If theangle of incidence precisely equals the critical angle θ_(C), then thelight wave is refracted along the refractive boundary. If the angle ofincidence is greater than the critical angle θ_(C), then the entirelight wave is reflected back into the cover glass without transmissioninto the atmosphere, according to the principle of total internalreflection (TIR). The reflected light waves are captured by one or morephotodetecting elements in the photodetecting/emitting layer 604. Inthis manner, the pixelated photoemitting element array communicates withthe pixelated photodetecting element array 624 through total internalreflection in the cover glass of the display 602.

Example light waves shown in FIG. 6 provide a more detailed illustrationof imaging of features of an object, such as ridges of a finger 601, onthe surface of the display 602. Light waves having an angle of incidencethat is less than the critical angle θ_(C) are transmitted through thecover glass within the angle shown by arrow 612. Such waves aresubstantially transmitted through the display surface 608, and anyportion of such waves that are reflected back through the cover glassare nominal in comparison to the light waves 614 and 616, which have anangle of incidence with the display surface 608 that is greater than thecritical angle θ_(C) and are reflected back into the glass through totalinternal reflection.

The light wave 614 intersects the display surface 608 at a point 618where there is no ridge of the fingerprint (i.e., there is a feature ofthe object at the point 618) in contact with the display surface 608. Assuch, the light wave 614 is reflected (as a non-feature portion of thereflected light) with substantially the same intensity back through thecover glass for capture by photodetecting elements in thephotodetecting/emitting layer 604.

In contrast, the light wave 616 intersects the display surface 608 at apoint 620 where there is a ridge of the fingerprint (i.e., a feature ofthe object) in contact with the display surface 608. The opticalcoupling at the point 620 of contact results in a weaker localizedreflected light signal at each photodetecting element (e.g., because offrustrated total internal reflection and/or other effects), in which thelight wave 616 is reflected in a scatter pattern 622 back through thecover glass for capture by photodetecting elements in thephotodetecting/emitting layer 604. The scatter pattern 622 results inthe intensity of reflected light from the light wave 616 at anyparticular location of the pixelated photodetecting element array 624being less than the captured light from the light wave 614. As such,light detection associated with the fingerprint ridges generallyexhibits a lower light intensity than light detection associated withthe fingerprint valleys. Image processing circuitry and/or software canfurther process the resulting composite image to invert the intensities(e.g., to provide a more tonally accurate image) or to otherwise enhancethe image.

FIG. 7 illustrates an example pixel array 700, wherein each pixelincludes a photoemitting element and a photodetecting element occupyingdistinct areas of the pixel. The pixel array 700 is controlled by sensorcontrol instructions via a pixel selector signal bus 702, which selectsthe row and column of each pixel that is selected to emit light forimaging and/or that is selected to detect light reflected from therefractive boundary at the display surface. It should be understood thatthe example pixels shown in FIG. 7 are a subset of the pixels that willtypically extend across the entire area of the display or across apredetermined area of the display (e.g., a fingerprint sensing region).

A pixel (such as pixels 704, 706, 708, 710, 712, and 714) includes aphotodetector including one or more photodetecting elements and aphotoemitter including one or more photoemitting elements (such as red,green, and blue photoemitting elements). Each photodetector shares pixelarea with a corresponding photoemitter, substantially in the same layerof the display, although individual photodetectors and photoemitters mayoccupy different layers in some implementations. In one implementation,a photodetector occupies a layer positioned below the photoemitter, andreflected light is transmitted through a cover glass of the display andthrough a transparent or translucent material or substrate binding thephotoemitters together in the pixel array 700. In other implementations,a photoemitter occupies a layer positioned below a photodetector of thesame pixel, and light is emitted through a transparent or translucentmaterial into the cover glass of the display. The reflected light isthen detected by the photodetector on the layer positioned above thephotoemitter.

Each pixel is also associated with driver electronics, which control theoperation of the photodetector and photoemitter of the pixel, includingwithout limitation interpreting row/column selector signals to determinewhich pixel is selected by sensor control instructions via a pixelselector signal bus and/or interpreting whether to turn the pixel on asa photoemitter or a photodetector. In one implementation, each pixel iscontrolled by a pixel selection signal communicated by single pixelselection signal bus input to the pixel. The pixel selector signal bus702, can include a portion of the bus that provides an address or apixelated row and/or column location in the photodetecting/emittingarrays of the display. For example, the pixel selection signal mayspecify one or more rows and one or more columns of pixels in the pixelarray 700, and each specified pixel turns on the photoemitter or thephotodetector as specified in the pixel selection signal.

In the illustrated implementation, each pixel includes a photodetectorand a separate photoemitter. In other implementations, a photoemittingelement may be reverse-biased to operate as a photodetecting element,such that a pixel includes a circuit that operates as either aphotoemitter or a photodetector, depending on the voltage applied to thecircuit. An output data signal component is communicated back to theimage processing circuitry and software of the system (e.g., via thepixel selector signal bus 702).

As shown in FIG. 7, the pixel 704 includes a photoemitter 716 and aphotodetector 718, and is coupled to driver electronics 720, which maybe outside or within the areal bounds of the pixel 704 (whether in thesame plane or a different plane). The driver electronics 720 receive thepixel selection signal from the pixel selection signal bus 702 andinterprets the pixel selection signal to determine whether the pixel 704is selected for operation and/or whether the photoemitter 716 or thephotodetector 718 is selected for operation. In one implementation, thephotoemitter versus photodetector selection is selected based on apolarity of a signal component on the pixel selection signal bus 702,although other photoemitter versus photodetector selection modes may beemployed. Similar pixels of the pixel array 700 are shown in FIG. 7 (thepixel 706 with a photoemitter 722 and a photodetector 724, associatedwith driver electronics 726; the pixel 708 with a photoemitter 730 and aphotodetector 733, associated with driver electronics 734; the pixel 710with a photoemitter 736 and a photodetector 738, associated with driverelectronics 740; the pixel 712 with a photoemitter 742 and aphotodetector 744, associated with driver electronics 746; and the pixel714 with a photoemitter 748 and a photodetector 750, associated withdriver electronics 752), and the pixel array 700 will typically includeother similarly configured pixels across a portion of the display oracross the entire display.

FIG. 8 illustrates an example pixel array 800, wherein each pixelincludes a photoemitting element and a photodetecting element occupyingoverlapping areas of the pixel. The pixel array 800 is controlled bysensor control instructions via a pixel selector signal bus 802, whichselects the row and column of each pixel that is selected to emit lightfor imaging and/or that is selected to detect light reflected from therefractive boundary at the display surface. It should be understood thatthe example pixels shown in FIG. 8 are a subset of the pixels that willtypically extend across the entire area of the display or across apredetermined area of the display (e.g., a fingerprint sensing region).

A pixel (such as pixels 804, 806, 808, 810, 812, 814, 854, and 856)includes a photodetector including one or more photodetecting elementsand a photoemitter including one or more photoemitting elements (such asred, green, and blue photoemitting elements). Each photodetectoroverlaps a corresponding photoemitter of the same pixel and occupies adifferent layer of the display. In the illustrated implementation, thephotodetector 818 occupies a layer positioned below the photoemitter816, and reflected light is transmitted through a cover glass of thedisplay and through a transparent or translucent material or substratebinding the photoemitters together in the pixel array 800. In otherimplementations, a photoemitter occupies a layer positioned below aphotodetector of the same pixel, and light is emitted through atransparent or translucent material into the cover glass of the display.The reflected light is then detected by the photodetector on the layerpositioned above the photoemitter.

Each pixel is also associated with driver electronics, which control theoperation of the photodetector and photoemitter of the pixel, includingwithout limitation interpreting row/column selector signals to determinewhich pixel is selected by sensor control instructions via a pixelselector signal bus and/or interpreting whether to turn the pixel on asa photoemitter or a photodetector. In one implementation, each pixel iscontrolled by a pixel selection signal communicated by single pixelselection signal bus input to the pixel. The pixel selector signal bus802, can include a portion of the bus that provides an address or apixelated row and/or column location in the photodetecting/emittingarrays of the display. For example, the pixel selection signal mayspecify one or more rows and one or more columns of pixels in the pixelarray 800, and each specified pixel turns on the photoemitter or thephotodetector as specified in the pixel selection signal.

In the illustrated implementation, each pixel includes a photodetectorand a separate photoemitter. In other implementations, a photoemittingelement may be reverse-biased to operate as a photodetecting element,such that a pixel includes a circuit that operates as either aphotoemitter or a photodetector, depending on the voltage applied to thecircuit. An output data signal component is communicated back to theimage processing circuitry and software of the system (e.g., via thepixel selector signal bus 702).

As shown in FIG. 8, the pixel 804 includes a photoemitter 816 and aphotodetector 818, and is coupled to driver electronics 820, which maybe outside or within the areal bounds of the pixel 804, whether in thesame plane or a different plane). The driver electronics 820 receive thepixel selection signal from the pixel selection signal bus 802 andinterprets the pixel selection signal to determine whether the pixel 804is selected for operation and/or whether the photoemitter 816 or thephotodetector 818 is selected for operation. In one implementation, thephotoemitter versus photodetector selection is selected based on apolarity of a signal component on the pixel selection signal bus 802,although other photoemitter versus photodetector selection modes may beemployed. Similar pixels of the pixel array 800 are shown in FIG. 8 (thepixel 806 with a photoemitter 822 and a photodetector 824, associatedwith driver electronics 826; the pixel 808 with a photoemitter 830 and aphotodetector 832, associated with driver electronics 834, the pixel 810with a photoemitter 836 and a photodetector 838, associated with driverelectronics 840; the pixel 812 with a photoemitter 842 and aphotodetector 844, associated with driver electronics 846; the pixel 814with a photoemitter 848 and a photodetector 850, associated with driverelectronics 852; the pixel 854 with a photoemitter 858 and aphotodetector 860, associated with driver electronics 862; and the pixel856 with a photoemitter 864 and a photodetector 866, associated withdriver electronics 868), and the pixel array 800 will typically includeother similarly configured pixels across a portion of the display oracross the entire display.

FIG. 9 illustrates a schematic of an example pixel driver electronicscircuit 900 for a pixel 902 having one or more photoemitting elements ina photoemitter 904 and one or more photodetecting elements in aphotodetector 906. Row and column selector signals are communicated tothe driver electronics 908 via a pixel selector bus 910, the columnselector signal including a component signal for selecting whether thepixel 902 should operate as a photodetector or a photoemitter. In theillustrated implementation, the polarity of a signal component of thecolumn selector bus provides this emitter/detector selectionfunctionality. If the signal component is positive, then a transistor T2of the photoemitter driver electronics 912 is turned on and a transistorT3 of the photodetector driver electronics 914 is turned off, causingthe photoemitter 904 of the pixel 902 to turn on and emit light and thephotodetector 906 of the pixel 902 to turn off. If the signal componentis negative, then the transistor T3 of the photodetector driverelectronics 914 is turned on and the transistor T2 of the photoemitterdriver electronics 912 is turned off, causing the photodetector 906 ofthe pixel 902 to turn on and detect light reflected from the cover glassdisplay surface interface by total internal reflection and causes thephotoemitter 914 of the pixel 902 to turn off.

The schematic of FIG. 9 illustrates an example of selecting whether asingle pixel, having both photodetecting and photoemitting elements, canbe controlled by a single pixel selector signal bus to operate as eithera photodetector or a photoemitter. Other single bus designs are alsocontemplated. Furthermore, in implementations in which a single elementof a pixel may be switched from a photoemitter to a photodetector (andvice versa) according to a bias voltage (e.g., reversing a voltage biascauses the element to switch to the opposite functionality), the biasvoltage can also be controlled by a signal component of the single pixelselector signal bus.

FIG. 10 illustrates a schematic of an example triggered image sensingsystem 1000 for a display of an electronic device. The display includesa cover glass 1002 for which at least a portion of the display area isoccupied by a photoemitter/detector array 1004 (e.g., any remainingdisplay may merely be occupied by a photoemitter array, without anyphotodetectors). A sensing trigger 1005, as example means for initiatingan imaging scan, is positioned to detect location of an action intendedto initiate an imaging scan operation, such as detection of a conductiveobject in proximity to the sensing trigger 1005, in the case of acapacitive sensor, for example. In an alternative implementation, thesensing trigger 1005 includes a pressure sensor to detect a location ofpressure applied to the region of the cover glass 1002 above the sensingtrigger 1005, such as pressure applied by a finger pressed against thecover glass 1002. Another example sensor for detecting the location tobe imaged may include a resistive sensor. In yet another alternativeimplementation, the triggering sensor may be embedded in the display ofthe electronic device without substantially adding thickness to thedisplay (e.g., within the same layer by within the bezel or by with auser-manipulated button in the bezel, the side of the electronic deviceor the back-side of the electronic device).

In such implementations, the sensing trigger 1005 allows the triggeredimage sensing system 1000 to refrain from scanning the display area withthe photoemitters and photodetectors until the imaging scan is triggeredby the sensing trigger 1005, thereby conserving power and processingresources. For example, upon sensing an initiating action intended toinitiate an imaging operation, such as a finger press on the displaysurface, the sensing trigger 1005 can transmit trigger data 1007 toimage processing circuitry and software 1008 to initiate an imagingscan. It should be noted that the sensing trigger 1005 may be inoverlapping proximity within the display, although other implementationsmay employ a separate sensing trigger 1005, such as a home button, apower button, or another display-based control. The display may presenta visible prompt on the display to indicate to a user the area of thedisplay that will be image scanned (e.g., the area on which to place afinger for fingerprint scanning).

Responsive to control signals 1016 provided by image processingcircuitry and software 1008, photoemitting elements in thephotoemitter/detector array 1004 emit scanning light through the coverglass 1002. Some portion of the scanning light is reflected from arefractive boundary at the surface 1010, and some portion of thereflected light is scattered off object features 1014 optically-coupledat the surface 1010 of the cover glass 1002. The reflected light,including the feature-scattered light, is captured by photodetectingelements in the photoemitter/detector array 1004 and transmitted to theimage processing circuitry and software 1008 from thephotoemitter/detector array 1004.

In one implementation, the imaging scan can also be localized to an areaof the display corresponding to the sensing trigger 1005, such thatmeans for localizing the imaging scan includes the sensing trigger 1005.For example, in a configuration in which the photoemitter/detector array1004 occupies a large area (or the entire area) of the display, thelocation at which the initiating action is detected can be communicatedto the image processing circuitry and software 1008 with the triggerdata 1007 (as location data) so that the image processing circuitry andsoftware 1008 can localize the image scanning to the area of the sensedinitiation action by limiting the control signals 1016 to acorresponding area of the photoemitter/detector array 1004.

In one implementation, the photoemitter/detector array 1004 includes a300 ppi sensor array having a sensing area of 1.0″×0.8″. At 8bits/pixel, the photoemitter/detector array 1004 can be calculated tocapture 576,000 bits/frame (i.e., (300×1.0)×(300×0.8)×8), although otherphotoemitter/detector array 1004 configurations may be employed.Assuming the above-described 576,000 bits/frame metric and a 60frame/sec scan rate, the photoemitter/detector array 1004 provides adata rate of 34,560,000 bits/sec of scanned image data 1006 being sentto the image processing circuitry and software 1008, which stitches thescans of captured light together into a composite image representing thescanned object (e.g., a fingerprint).

FIG. 11 illustrates example operations 1100 for image sensing with adisplay. An emitting operation 1102 emits a light signal through a coverglass of a display to a display surface of the display. The light signalis emitted from one or more selected photoemitting elements of apixelated photoemitting element array of the display. A captureoperation 1104 captures a reflected light signal at individualphotodetecting elements of a pixelated photodetecting element arraypositioned relative to the pixelated photoemitting element array and thecover glass to receive the reflected light signal. The reflected lightsignal includes a portion of the emitted light signal reflected by totalinternal reflection from a refractive boundary at the display surface ofthe cover glass. A stitching operation 1106 stitches the capturedreflected light signal received by each photodetecting element of thephotodetecting element array into a composite image of an object incontact with the display surface of the display.

FIG. 12 illustrates example operations 1200 for switching a pixelbetween photoemitting mode and photodetecting mode. A communicationoperation 1202 communicates a pixel selection signal component on apixel selector signal bus. A switching operation 1204 switches operationof each pixel in a pixel array in a display between a photoemittingoperation and a photodetecting operation by the pixel selector signalcomponent received from the pixel selector signal bus. Each pixel in thepixel array includes one or more photodetectors and one or morephotoemitting elements. The pixel array is electrically connected to thepixel selector signal bus and includes multiple pixels configured tosense an image from a surface of the display.

FIG. 13 illustrates example operations 1300 for triggered image sensingwith a display. An emitting operation 1302 emits a light signal througha cover glass of a display to a display surface of the display. Thelight signal is emitted from one or more selected photoemitting elementsof a pixelated photoemitting element array of the display as part of animaging scan. A capturing operation 1304 captures a reflected lightsignal at individual photodetecting elements of a pixelatedphotodetecting element array positioned relative to a pixelatedphotoemitting element array and the cover glass to receive the reflectedlight signal as part of the imaging scan. A triggering operation 1306triggers the imaging scan by the pixelated photoemitting element arrayand the pixelated photodetecting element array, responsive to detectionof an initiating action by a sensing trigger positioned relative to thepixelated photodetecting element array. A stitching operation 1308stitches the captured reflected light signal received by eachphotodetecting element of the photodetecting element array into acomposite image of an object in contact with the display surface of thedisplay.

FIG. 14 illustrates an example processing system 1400 for use in anelectronic device for image sensing and/or triggered image sensing on adisplay 1406. The processing system 1400 includes one or more processorunits 1402 (discrete or integrated microelectronic chips and/or separatebut integrated processor cores), at least one memory device 1404 (whichmay be integrated into systems or chips of the processing system 1400),the display 1406 (e.g., a touchscreen display, an OLED display withphotodetectors, etc.), and other interfaces 1408 (e.g., a keyboardinterface). The memory device 1404 generally includes both volatilememory (e.g., RAM) and non-volatile memory (e.g., flash memory). Anoperating system 1410, such as one of the varieties of the MicrosoftWindows® operating system, resides in the memory device 1404 and isexecuted by at least one of the processor units 1402, although it shouldbe understood that other operating systems may be employed. Otherfeatures of the electronic device 1400 may include without limitation aphotodetecting/photoemitting layer in the display, a pixel selectorsignal bus, and a sensing trigger (e.g., a pressure sensor, a proximitysensor, etc.).

One or more applications 1412, such as image scanning software,triggering software, sensor control instructions, etc., are loaded inthe memory device 1404 and executed on the operating system 1410 by atleast one of the processor units 1402. The processing system 1400includes a power supply 1416, which is powered by one or more batteriesand/or other power sources and which provides power to other componentsof the processing system 1400. The power supply 1416 may also beconnected to an external power source that overrides or recharges thebuilt-in batteries or other power sources.

The processing system 1400 includes one or more communicationtransceivers 1430 to provide network connectivity (e.g., mobile phonenetwork, Wi-Fi®, BlueTooth®, etc.). The processing system 1400 alsoincludes various other components, such as a positioning system 1420(e.g., a global positioning satellite transceiver), one or moreaccelerometers 1422, one or more cameras 1424, one or more audiointerfaces (e.g., an audio interface, such a microphone, an audioamplifier and speaker and/or audio jack), one or more antennas (1432),and additional storage 1428. Other configurations may also be employed.

In an example implementation, a mobile operating system, variousapplications, modules for image scanning, triggered image scanning,image stitching, image recognition (e.g., fingerprint recognition),device access control, security, and other modules and services may beembodied by instructions stored in the memory device 1404 and/or storagedevices 1428 and processed by the processing unit 1402. Security andaccess control parameters, training fingerprint patterns, and other datamay be stored in the memory device 1404 and/or storage devices 1428 aspersistent datastores.

An example imaging system includes a cover glass having a displaysurface and a pixelated photoemitting element array. One or moreselected photoemitting elements of the pixelated photoemitting elementarray are configured to emit a light signal through the cover glass tothe display surface. The example imaging system also includes apixelated photodetecting element array positioned relative to thepixelated photoemitting element array and the cover glass to receive areflected light signal at individual photodetecting elements of thepixelated photodetecting element array. The reflected light signalincludes a portion of the emitted light signal reflected by totalinternal reflection from a refractive boundary at the display surface ofthe cover glass.

Another example imaging system of any preceding system is configuredsuch that the reflected light signal excludes a portion of the emittedlight signal transmitted through the refractive boundary at the displaysurface.

Another example imaging system of any preceding system is configuredsuch that the reflected light signal excludes a portion of the emittedlight signal transmitted through the refractive boundary at the displaysurface. The transmitted portion of the emitted light signal has anangle of incidence with the display surface that is less than a criticalangle of the refractive boundary at the display surface of the coverglass.

Another example imaging system of any preceding system is configuredsuch that the reflected light signal includes a portion of the emittedlight signal reflected by total internal reflection. The reflectedportion of the emitted light signal has an angle of incidence with thedisplay surface that is greater than a critical angle of the refractiveboundary at the display surface of the cover glass.

Another example imaging system of any preceding system is configuredsuch that the reflected light signal includes a feature-scatteredportion of the emitted light signal resulting from total internalreflection from the refractive boundary at the display surface of thecover glass. The feature-scattered portion of the emitted light signalcorresponds to a region of optical coupling at the refractive boundaryat the display surface of the cover glass and an optically-coupledfeature of an object on the display surface of the cover glass.

Another example imaging system of any preceding system is configuredsuch that the reflected light signal includes a non-feature portion ofthe emitted light signal resulting from total internal reflection fromthe refractive boundary at the display surface of the covered glass. Thenon-feature portion of the emitted light signal corresponds to a regionof the display surface of the cover glass in which a feature of anobject is not optically coupled at the refractive boundary at thedisplay surface of the cover glass.

Another example imaging system of any preceding system further includesimaging processing circuitry electronically connected to the pixelatedphotodetecting element array and configured to stitch the reflectedlight signal received by each photodetecting element of the pixelatedphotodetecting element array into a composite image of an object incontact with the display surface of the display.

Another example imaging system of any preceding system further includesimaging processing circuitry electronically connected to the pixelatedphotoemitting element array and the pixelated photodetecting elementarray and configured to scan emitted light from an area of the pixelatedphotoemitting element array and to capture by the pixelatedphotodetecting element array the scanned emitted light as the reflectedlight signal as the scanned emitted light reflects from the refractiveboundary at the display surface of the cover glass.

An example method includes emitting a light signal through a cover glassof a display to a display surface of the display. The light signal isemitted from one or more selected photoemitting elements of a pixelatedphotoemitting element array of the display. The method further includescapturing a reflected light signal at individual photodetecting elementsof a pixelated photodetecting element array positioned relative to thepixelated photoemitting element array and the cover glass to receive thereflected light signal. The reflected light signal includes a portion ofthe emitted light signal reflected by total internal reflection from arefractive boundary at the display surface of the cover glass.

Another example method of any preceding method is operated such that thereflected light signal excludes a portion of the emitted light signaltransmitted through the refractive boundary at the display surface.

Another example method of any preceding method is operated such that thereflected light signal excludes a portion of the emitted light signaltransmitted through the refractive boundary at the display surface. Thetransmitted portion of the emitted light signal has an angle ofincidence with the display surface that is less than a critical angle ofthe refractive boundary at the display surface of the cover glass.

Another example method of any preceding method is operated such that thereflected light signal includes a portion of the emitted light signalreflected by total internal reflection. The reflected portion of theemitted light signal has an angle of incidence with the display surfacethat is greater than a critical angle of the refractive boundary at thedisplay surface of the cover glass.

Another example method of any preceding method is operated such that thereflected light signal includes a feature-scattered portion of theemitted light signal resulting from total internal reflection from therefractive boundary at the display surface of the covered glass, thefeature-scattered portion of the emitted light signal corresponding to aregion of optical coupling at the refractive boundary at the displaysurface of the cover glass and an optically-coupled feature of an objecton the display surface of the cover glass.

Another example method of any preceding method is operated such that thereflected light signal includes a non-feature portion of the emittedlight signal resulting from total internal reflection from therefractive boundary at the display surface of the covered glass, thenon-feature portion of the emitted light signal corresponding to aregion of the display surface of the cover glass in which a feature ofan object is not optically coupled at the refractive boundary at thedisplay surface of the cover glass.

Another example method of any preceding method further includingstitching the captured reflected light signal received by eachphotodetecting element of the photodetecting element array into acomposite image of an object in contact with the display surface of thedisplay.

Another example method of any preceding method further includingscanning emitted light from an area of the pixelated photoemittingelement array and to capture by the pixelated photodetecting elementarray the scanned emitted light as the reflected light signal as thescanned emitted light reflects from the refractive boundary at thedisplay surface of the cover glass.

An example electronic device includes a cover glass having a displaysurface and a pixelated photoemitting element array. One or moreselected photoemitting elements of the pixelated photoemitting elementarray are configured to emit a light signal through the cover glass tothe display surface. The example electronic device also includes apixelated photodetecting element array positioned relative to thepixelated photoemitting element array and the cover glass to receive areflected light signal at individual photodetecting elements of thepixelated photodetecting element array. The reflected light signalincludes a portion of the emitted light signal reflected by totalinternal reflection from a refractive boundary at the display surface ofthe cover glass. The example electronic device also includes imageprocessing circuitry electrically coupled to the pixelated photoemittingelement array and the pixelated photoemitting element array andconfigured to stitch the reflected light signal received by eachphotodetecting element of the pixelated photodetecting element arrayinto a composite image of an object in contact with the display surfaceof the display.

Another example electronic device of any preceding device is configuredsuch that the reflected light signal excludes a portion of the emittedlight signal transmitted through the refractive boundary at the displaysurface. The transmitted portion of the emitted light signal has anangle of incidence with the display surface that is less than a criticalangle of the refractive boundary at the display surface of the coverglass.

Another example electronic device of any preceding device is configuredsuch that the reflected light signal includes a feature-scatteredportion of the emitted light signal resulting from total internalreflection from the refractive boundary at the display surface of thecover glass. The feature-scattered portion of the emitted light signalcorresponds to a region of optical coupling at the refractive boundaryat the display surface of the cover glass and an optically-coupledfeature of an object on the display surface of the cover glass.

Another example electronic device of any preceding device is configuredsuch that the reflected light signal includes a non-feature portion ofthe emitted light signal resulting from total internal reflection fromthe refractive boundary at the display surface of the covered glass. Thenon-feature portion of the emitted light signal corresponds to a regionof the display surface of the cover glass in which a feature of anobject is not optically coupled at the refractive boundary at thedisplay surface of the cover glass.

An example system includes means for emitting a light signal through acover glass of a display to a display surface of the display. The lightsignal is emitted from one or more selected photoemitting elements of apixelated photoemitting element array of the display. The method furtherincludes means for capturing a reflected light signal at individualphotodetecting elements of a pixelated photodetecting element arraypositioned relative to the pixelated photoemitting element array and thecover glass to receive the reflected light signal. The reflected lightsignal includes a portion of the emitted light signal reflected by totalinternal reflection from a refractive boundary at the display surface ofthe cover glass.

Another example system of any preceding system is configured such thatthe reflected light signal excludes a portion of the emitted lightsignal transmitted through the refractive boundary at the displaysurface.

Another example system of any preceding system is configured such thatthe reflected light signal excludes a portion of the emitted lightsignal transmitted through the refractive boundary at the displaysurface. The transmitted portion of the emitted light signal has anangle of incidence with the display surface that is less than a criticalangle of the refractive boundary at the display surface of the coverglass.

Another example system of any preceding system is configured such thatthe reflected light signal includes a portion of the emitted lightsignal reflected by total internal reflection. The reflected portion ofthe emitted light signal has an angle of incidence with the displaysurface that is greater than a critical angle of the refractive boundaryat the display surface of the cover glass.

Another example system of any preceding system is configured such thatthe reflected light signal includes a feature-scattered portion of theemitted light signal resulting from total internal reflection from therefractive boundary at the display surface of the covered glass, thefeature-scattered portion of the emitted light signal corresponding to aregion of optical coupling at the refractive boundary at the displaysurface of the cover glass and an optically-coupled feature of an objecton the display surface of the cover glass.

Another example system of any preceding system is configured such thatthe reflected light signal includes a non-feature portion of the emittedlight signal resulting from total internal reflection from therefractive boundary at the display surface of the covered glass, thenon-feature portion of the emitted light signal corresponding to aregion of the display surface of the cover glass in which a feature ofan object is not optically coupled at the refractive boundary at thedisplay surface of the cover glass.

Another example system of any preceding system further includes meansfor stitching the captured reflected light signal received by eachphotodetecting element of the photodetecting element array into acomposite image of an object in contact with the display surface of thedisplay.

Another example system of any preceding system further includes meansfor scanning emitted light from an area of the pixelated photoemittingelement array and to capture by the pixelated photodetecting elementarray the scanned emitted light as the reflected light signal as thescanned emitted light reflects from the refractive boundary at thedisplay surface of the cover glass.

Another example imaging system includes a pixel selector signal busconfigured to communicate a pixel selection signal component and a pixelarray of a display. The pixel array is electrically connected to thepixel selector signal bus and includes multiple pixels configured tosense an image of an object in contact with a surface of the display.Each pixel in the pixel array includes one or more photodetectingelements and one or more photoemitting elements. Operation of each pixelis switched between the one or more photodetecting elements and the oneor more photoemitting elements by the pixel selector signal componentreceived from the pixel selector signal bus.

Another example imaging system of any preceding system is configuredsuch that the one or more photodetectors and the one or morephotoemitting elements are positioned within the display.

Another example imaging system of any preceding system is configuredsuch that the one or more photodetectors and the one or morephotoemitting elements are bonded to the display opposite a displaysurface of a cover glass of the display.

Another example imaging system of any preceding system is configuredsuch that the one or more photodetecting elements are bonded to thedisplay opposite a display surface of a cover glass of the display andthe one or more photoemitting elements are positioned within thedisplay.

Another example imaging system of any preceding system is configuredsuch that the one or more photodetecting elements are positioned withinthe display and the one or more photoemitting elements are bonded to thedisplay opposite a display surface of a cover glass of the display.

Another example imaging system of any preceding system is configuredsuch that the polarity of the pixel selector signal component receivedfrom the pixel selector signal bus selects photoemitter driverelectronics to turn on the one or more photoemitting elements in thepixel.

Another example imaging system of any preceding system is configuredsuch that the polarity of the pixel selector signal component receivedfrom the pixel selector signal bus selects photodetector driverelectronics to turn on the one or more photodetecting elements in thepixel.

Another example imaging system of any preceding system is configuredsuch that the pixel selector signal bus communicates an output datasignal component from the pixel when the photodetector driverelectronics has turned on the one or more photodetecting elements in thepixel.

Another example method includes communicating a pixel selection signalcomponent on a pixel selector signal bus and switching operation of eachpixel of a pixel array in a display between a photoemitting operationand a photodetecting operation by the pixel selector signal componentreceived from the pixel selector signal bus. Each pixel in the pixelarray includes one or more photodetectors and one or more photoemittingelements. The pixel array is electrically connected to the pixelselector signal bus and includes multiple pixels configured to sense animage of an object in contact with a surface of the display.

Another example method of any preceding method is operated such that theone or more photodetectors and the one or more photoemitting elementsare positioned within the display.

Another example method of any preceding method is operated such that theone or more photodetectors and the one or more photoemitting elementsare bonded to the display opposite a display surface of a cover glass ofthe display.

Another example method of any preceding method is operated such that theone or more photodetecting elements are bonded to the display opposite adisplay surface of a cover glass of the display and the one or morephotoemitting elements are positioned within the display.

Another example method of any preceding method is operated such that theone or more photodetecting elements are positioned within the displayand the one or more photoemitting elements are bonded to the displayopposite a display surface of a cover glass of the display.

Another example method of any preceding method is operated such that thepolarity of the pixel selector signal component received from the pixelselector signal bus selects photoemitter driver electronics to turn onthe one or more photoemitting elements in the pixel.

Another example method of any preceding method is operated such that thepolarity of the pixel selector signal component received from the pixelselector signal bus selects photodetector driver electronics to turn onthe one or more photodetecting elements in the pixel.

Another example method of any preceding method is operated such that thepixel selector signal bus communicates an output data signal componentfrom the pixel when the photodetector driver electronics has turned onthe one or more photodetecting elements in the pixel.

Another example electronic device includes a cover glass of a displayhaving a display surface and a pixel selector signal bus configured tocommunicate a pixel selection signal component. The example electronicdevice also includes a pixel array of the display. The pixel array iselectrically connected to the pixel selector signal bus and includesmultiple pixels configured to sense an image of an object in contactwith a surface of the display. Each pixel in the pixel array includesone or more photodetecting elements and one or more photoemittingelements. Operation of each pixel is switched between the one or morephotodetecting elements and the one or more photoemitting elements bythe pixel selector signal component received from the pixel selectorsignal bus. The example electronic device also includes image processingcircuitry electrically coupled to the pixel array and configured to scanlight from multiple pixels of the pixel array and stitch a light signalreflected from a refractive boundary at the display surface and receivedby photodetecting elements of the pixelated photodetecting element arrayinto a composite image of the object.

Another example electronic device of any preceding device is configuredsuch that the one or more photodetectors and the one or morephotoemitting elements are positioned within the display.

Another example electronic device of any preceding device is configuredsuch that the one or more photodetecting elements and the one or morephotoemitting elements are positioned on different layers within thedisplay.

Another example electronic device of any preceding device is configuredsuch that the polarity of the pixel selector signal component receivedfrom the pixel selector signal bus alternatively selects photoemitterdriver electronics to turn on the one or more photoemitting elements inthe pixel and selects photodetector driver electronics to turn on theone or more photodetecting elements in the pixel.

Another example system includes means for communicating a pixelselection signal component on a pixel selector signal bus and means forswitching operation of each pixel of a pixel array in a display betweena photoemitting operation and a photodetecting operation by the pixelselector signal component received from the pixel selector signal bus.Each pixel in the pixel array includes one or more photodetectors andone or more photoemitting elements. The pixel array is electricallyconnected to the pixel selector signal bus and includes multiple pixelsconfigured to sense an image of an object in contact with a surface ofthe display.

Another example system of any preceding system is configured such thatthe one or more photodetectors and the one or more photoemittingelements are positioned within the display.

Another example system of any preceding system is configured such thatthe one or more photodetectors and the one or more photoemittingelements are bonded to the display opposite a display surface of a coverglass of the display.

Another example system of any preceding system is configured such thatthe one or more photodetecting elements are bonded to the displayopposite a display surface of a cover glass of the display and the oneor more photoemitting elements are positioned within the display.

Another example system of any preceding system is configured such thatthe one or more photodetecting elements are positioned within thedisplay and the one or more photoemitting elements are bonded to thedisplay opposite a display surface of a cover glass of the display.

Another example system of any preceding system is configured such thatthe polarity of the pixel selector signal component received from thepixel selector signal bus selects photoemitter driver electronics toturn on the one or more photoemitting elements in the pixel.

Another example system of any preceding system is configured such thatthe polarity of the pixel selector signal component received from thepixel selector signal bus selects photodetector driver electronics toturn on the one or more photodetecting elements in the pixel.

Another example system of any preceding system is configured such thatthe pixel selector signal bus communicates an output data signalcomponent from the pixel when the photodetector driver electronics hasturned on the one or more photodetecting elements in the pixel.

Another example imaging system includes a cover glass having a displaysurface and a pixelated photoemitting element array. One or moreselected photoemitting elements of the pixelated photoemitting elementarray is configured to emit a light signal through the cover glass tothe display surface as part of an imaging scan. The example imagingsystem also includes a pixelated photodetecting element array positionedrelative to the pixelated photoemitting element array and the coverglass to receive a reflected light signal at individual photodetectingelements of the pixelated photodetecting element array as part of theimaging scan. The example imaging system also includes a sensing triggerconfigured to trigger the imaging scan by the pixelated photoemittingelement array and the pixelated photodetecting element array, responsiveto detection of an initiating action by the sensing trigger.

Another example imaging system of any preceding system is configuredsuch that the sensing trigger includes a pressure sensor.

Another example imaging system of any preceding system is configuredsuch the sensing trigger includes a capacitive sensor.

Another example imaging system of any preceding system is configuredsuch the sensing trigger includes a resistive sensor.

Another example imaging system of any preceding system is configuredsuch the reflected light signal received at individual photodetectingelements of the pixelated photodetecting element array is reflected froma refractive boundary at the display surface of the cover glass by totalinternal reflection.

Another example imaging system of any preceding system further includesimage processing circuitry coupled to the sensing trigger, the pixelatedphotoemitting element array and the pixelated photodetecting elementarray. The sensing trigger transmits trigger data to image processingcircuitry to initiate an imaging scan through the cover glass.

Another example imaging system of any preceding system further includesimage processing circuitry coupled to the sensing trigger, the pixelatedphotoemitting element array and the pixelated photodetecting elementarray. The sensing trigger transmits location information in the triggerdata to the image processing circuitry to localize the imaging scanthrough the cover glass.

Another example method includes emitting a light signal through a coverglass of a display to a display surface of the display. The light signalis emitted from one or more selected photoemitting elements of apixelated photoemitting element array of the display as part of animaging scan. The example method also includes capturing a reflectedlight signal at individual photodetecting elements of a pixelatedphotodetecting element array positioned relative to a pixelatedphotoemitting element array and the cover glass to receive the reflectedlight signal as part of the imaging scan. The example method alsoincludes triggering the imaging scan by the pixelated photoemittingelement array and the pixelated photodetecting element array, responsiveto detection of an initiating action by a sensing trigger positionedrelative to the pixelated photodetecting element array.

Another example method of any preceding method is operated such that thesensing trigger includes a pressure sensor.

Another example method of any preceding method is operated such that thesensing trigger includes a capacitive sensor.

Another example method of any preceding method is operated such that thesensing trigger includes a resistive sensor.

Another example method of any preceding method is operated such that thereflected light signal received at individual photodetecting elements ofthe pixelated photodetecting element array is reflected from arefractive boundary at the display surface of the cover glass by totalinternal reflection.

Another example method of any preceding method further includesinitiating the imaging scan through the cover glass responsive toreceipt of triggering data transmitted by the imaging sensor andreceived by image processing circuitry.

Another example method of any preceding method further includeslocalizing the imaging scan through the cover glass responsive toreceipt of location information in the trigger data transmitted by theimaging sensor and received by the image processing circuitry.

Another example electronic device includes a cover glass having adisplay surface and a pixelated photoemitting element array. One or moreselected photoemitting elements of the pixelated photoemitting elementarray is configured to emit a light signal through the cover glass tothe display surface as part of an imaging scan. The example electronicdevice also includes a pixelated photodetecting element array positionedrelative to the pixelated photoemitting element array and the coverglass to receive a reflected light signal at individual photodetectingelements of the pixelated photodetecting element array as part of theimaging scan. The electronic device also includes a sensing triggerconfigured to trigger the imaging scan by the pixelated photoemittingelement array and the pixelated photodetecting element array, responsiveto detection of an initiating action by the sensing trigger. The exampleelectronic device also includes image processing circuitry coupled tothe sensing trigger, the pixelated photoemitting element array and thepixelated photodetecting element array. The sensing trigger transmitstrigger data to image processing circuitry to initiate the imaging scanthrough the cover glass.

Another example electronic device of any preceding device is configuredsuch that the sensing trigger includes a pressure sensor.

Another example electronic device of any preceding device is configuredsuch that the sensing trigger includes a capacitive sensor.

Another example electronic device of any preceding device is configuredsuch that the sensing trigger includes a resistive sensor.

Another example electronic device of any preceding device is configuredsuch that the reflected light signal received at individualphotodetecting elements of the pixelated photodetecting element array isreflected from a refractive boundary at the display surface of the coverglass by total internal reflection.

Another example electronic device of any preceding device is configuredsuch that the image processing circuitry is coupled to the sensingtrigger, the pixelated photoemitting element array and the pixelatedphotodetecting element array. The sensing trigger transmits locationinformation in the trigger data to the image processing circuitry tolocalize the imaging scan through the cover glass.

Another example system includes means for emitting a light signalthrough a cover glass of a display to a display surface of the display.The light signal is emitted from one or more selected photoemittingelements of a pixelated photoemitting element array of the display aspart of an imaging scan. The example system also includes means forcapturing a reflected light signal at individual photodetecting elementsof a pixelated photodetecting element array positioned relative to apixelated photoemitting element array and the cover glass to receive thereflected light signal as part of the imaging scan. The example systemalso includes means for triggering the imaging scan by the pixelatedphotoemitting element array and the pixelated photodetecting elementarray, responsive to detection of an initiating action by a sensingtrigger positioned relative to the pixelated photodetecting elementarray.

Another example system of any preceding system is configured such thatthe sensing trigger includes a pressure sensor.

Another example system of any preceding system is configured such thatthe sensing trigger includes a capacitive sensor.

Another example system of any preceding system is configured such thatthe sensing trigger includes a resistive sensor.

Another example system of any preceding system is configured such thatthe reflected light signal received at individual photodetectingelements of the pixelated photodetecting element array is reflected froma refractive boundary at the display surface of the cover glass by totalinternal reflection.

Another example system of any preceding system further includes meansfor initiating the imaging scan through the cover glass responsive toreceipt of triggering data transmitted by the imaging sensor andreceived by image processing circuitry.

Another example system of any preceding system further includes meansfor localizing the imaging scan through the cover glass responsive toreceipt of location information in the trigger data transmitted by theimaging sensor and received by the image processing circuitry.

The processing system 1400 may include a variety of tangiblecomputer-readable storage media and intangible computer-readablecommunication signals. Tangible computer-readable storage can beembodied by any available media that can be accessed by the processingsystem 1400 and includes both volatile and nonvolatile storage media,removable and non-removable storage media. Tangible computer-readablestorage media excludes intangible communications signals and includesvolatile and nonvolatile, removable and non-removable storage mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, program modules orother data. Tangible computer-readable storage media includes, but isnot limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CDROM, digital versatile disks (DVD) or other optical diskstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other tangible medium which canbe used to store the desired information and which can be accessed bythe processing system 1400. In contrast to tangible computer-readablestorage media, intangible computer-readable communication signals mayembody computer readable instructions, data structures, program modulesor other data resident in a modulated data signal, such as a carrierwave or other signal transport mechanism. The term “modulated datasignal” means a signal that has one or more of its characteristics setor changed in such a manner as to encode information in the signal. Byway of example, and not limitation, intangible communication signalsinclude signals traveling through wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media.

Some embodiments may comprise an article of manufacture. An article ofmanufacture may comprise a tangible storage medium to store logic.Examples of a storage medium may include one or more types ofcomputer-readable storage media capable of storing electronic data,including volatile memory or non-volatile memory, removable ornon-removable memory, erasable or non-erasable memory, writeable orre-writeable memory, and so forth. Examples of the logic may includevarious software elements, such as software components, programs,applications, computer programs, application programs, system programs,machine programs, operating system software, middleware, firmware,software modules, routines, subroutines, operation segments, methods,procedures, software interfaces, application program interfaces (API),instruction sets, computing code, computer code, code segments, computercode segments, words, values, symbols, or any combination thereof. Inone embodiment, for example, an article of manufacture may storeexecutable computer program instructions that, when executed by acomputer, cause the computer to perform methods and/or operations inaccordance with the described embodiments. The executable computerprogram instructions may include any suitable type of code, such assource code, compiled code, interpreted code, executable code, staticcode, dynamic code, and the like. The executable computer programinstructions may be implemented according to a predefined computerlanguage, manner or syntax, for instructing a computer to perform acertain operation segment. The instructions may be implemented using anysuitable high-level, low-level, object-oriented, visual, compiled and/orinterpreted programming language.

The implementations described herein are implemented as logical steps inone or more computer systems. The logical operations may be implemented(1) as a sequence of processor-implemented steps executing in one ormore computer systems and (2) as interconnected machine or circuitmodules within one or more computer systems. The implementation is amatter of choice, dependent on the performance requirements of thecomputer system being utilized. Accordingly, the logical operationsmaking up the implementations described herein are referred to variouslyas operations, steps, objects, or modules. Furthermore, it should beunderstood that logical operations may be performed in any order, unlessexplicitly claimed otherwise or a specific order is inherentlynecessitated by the claim language.

What is claimed is:
 1. An imaging system comprising: a cover glasshaving a display surface; a pixelated photoemitting element array, oneor more selected photoemitting elements of the pixelated photoemittingelement array being configured to emit a light signal through the coverglass to the display surface; and a pixelated photodetecting elementarray positioned relative to the pixelated photoemitting element arrayand the cover glass to receive a reflected light signal at individualphotodetecting elements of the pixelated photodetecting element array,the reflected light signal including a portion of the emitted lightsignal reflected by total internal reflection from a refractive boundaryat the display surface of the cover glass.
 2. The imaging system ofclaim 1 wherein the reflected light signal excludes a portion of theemitted light signal transmitted through the refractive boundary at thedisplay surface.
 3. The imaging system of claim 1 wherein the reflectedlight signal excludes a portion of the emitted light signal transmittedthrough the refractive boundary at the display surface, the transmittedportion of the emitted light signal having an angle of incidence withthe display surface that is less than a critical angle of the refractiveboundary at the display surface of the cover glass.
 4. The imagingsystem of claim 1 wherein the reflected light signal includes a portionof the emitted light signal reflected by total internal reflection, thereflected portion of the emitted light signal having an angle ofincidence with the display surface that is greater than a critical angleof the refractive boundary at the display surface of the cover glass. 5.The imaging system of claim 1 wherein the reflected light signalincludes a feature-scattered portion of the emitted light signalresulting from total internal reflection from the refractive boundary atthe display surface of the cover glass, the feature-scattered portion ofthe emitted light signal corresponding to a region of optical couplingat the refractive boundary at the display surface of the cover glass andan optically-coupled feature of an object on the display surface of thecover glass.
 6. The imaging system of claim 1 wherein the reflectedlight signal includes a non-feature portion of the emitted light signalresulting from total internal reflection from the refractive boundary atthe display surface of the covered glass, the non-feature portion of theemitted light signal corresponding to a region of the display surface ofthe cover glass in which a feature of an object is not optically coupledat the refractive boundary at the display surface of the cover glass. 7.The imaging system of claim 1 further comprising: imaging processingcircuitry electronically connected to the pixelated photodetectingelement array and configured to stitch the reflected light signalreceived by each photodetecting element of the pixelated photodetectingelement array into a composite image of an object in contact with thedisplay surface of the display.
 8. The imaging system of claim 1 furthercomprising: imaging processing circuitry electronically connected to thepixelated photoemitting element array and the pixelated photodetectingelement array and configured to scan emitted light from an area of thepixelated photoemitting element array and to capture by the pixelatedphotodetecting element array the scanned emitted light as the reflectedlight signal as the scanned emitted light reflects from the refractiveboundary at the display surface of the cover glass.
 9. A methodcomprising: emitting a light signal through a cover glass of a displayto a display surface of the display, the light signal being emitted fromone or more selected photoemitting elements of a pixelated photoemittingelement array of the display; and capturing a reflected light signal atindividual photodetecting elements of a pixelated photodetecting elementarray positioned relative to the pixelated photoemitting element arrayand the cover glass to receive the reflected light signal, the reflectedlight signal including a portion of the emitted light signal reflectedby total internal reflection from a refractive boundary at the displaysurface of the cover glass.
 10. The method of claim 9 wherein thereflected light signal excludes a portion of the emitted light signaltransmitted through the refractive boundary at the display surface. 11.The method of claim 9 wherein the reflected light signal excludes aportion of the emitted light signal transmitted through the refractiveboundary at the display surface, the transmitted portion of the emittedlight signal having an angle of incidence with the display surface thatis less than a critical angle of the refractive boundary at the displaysurface of the cover glass.
 12. The method of claim 9 wherein thereflected light signal includes a portion of the emitted light signalreflected by total internal reflection, the reflected portion of theemitted light signal having an angle of incidence with the displaysurface that is greater than a critical angle of the refractive boundaryat the display surface of the cover glass.
 13. The method of claim 9wherein the reflected light signal includes a feature-scattered portionof the emitted light signal resulting from total internal reflectionfrom the refractive boundary at the display surface of the coveredglass, the feature-scattered portion of the emitted light signalcorresponding to a region of optical coupling at the refractive boundaryat the display surface of the cover glass and an optically-coupledfeature of an object on the display surface of the cover glass.
 14. Themethod of claim 9 wherein the reflected light signal includes anon-feature portion of the emitted light signal resulting from totalinternal reflection from the refractive boundary at the display surfaceof the covered glass, the non-feature portion of the emitted lightsignal corresponding to a region of the display surface of the coverglass in which a feature of an object is not optically coupled at therefractive boundary at the display surface of the cover glass.
 15. Themethod of claim 9 further comprising: stitching the captured reflectedlight signal received by each photodetecting element of thephotodetecting element array into a composite image of an object incontact with the display surface of the display.
 16. The method of claim9 further comprising: scanning emitted light from an area of thepixelated photoemitting element array and to capture by the pixelatedphotodetecting element array the scanned emitted light as the reflectedlight signal as the scanned emitted light reflects from the refractiveboundary at the display surface of the cover glass.
 17. An electronicdevice comprising: a cover glass having a display surface; a pixelatedphotoemitting element array, one or more selected photoemitting elementsof the pixelated photoemitting element array being configured to emit alight signal through the cover glass to the display surface; a pixelatedphotodetecting element array positioned relative to the pixelatedphotoemitting element array and the cover glass to receive a reflectedlight signal at individual photodetecting elements of the pixelatedphotodetecting element array, the reflected light signal including aportion of the emitted light signal reflected by total internalreflection from a refractive boundary at the display surface of thecover glass; and image processing circuitry electrically coupled to thepixelated photoemitting element array and the pixelated photoemittingelement array and configured to stitch the reflected light signalreceived by each photodetecting element of the pixelated photodetectingelement array into a composite image of an object in contact with thedisplay surface of the display.
 18. The electronic device of claim 17wherein the reflected light signal excludes a portion of the emittedlight signal transmitted through the refractive boundary at the displaysurface, the transmitted portion of the emitted light signal having anangle of incidence with the display surface that is less than a criticalangle of the refractive boundary at the display surface of the coverglass.
 19. The electronic device of claim 17 wherein the reflected lightsignal includes a feature-scattered portion of the emitted light signalresulting from total internal reflection from the refractive boundary atthe display surface of the cover glass, the feature-scattered portion ofthe emitted light signal corresponding to a region of optical couplingat the refractive boundary at the display surface of the cover glass andan optically-coupled feature of an object on the display surface of thecover glass.
 20. The electronic device of claim 17 wherein the reflectedlight signal includes a non-feature portion of the emitted light signalresulting from total internal reflection from the refractive boundary atthe display surface of the covered glass, the non-feature portion of theemitted light signal corresponding to a region of the display surface ofthe cover glass in which a feature of an object is not optically coupledat the refractive boundary at the display surface of the cover glass.